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International application No. 

PCT/EP 99/00425 


International filing date (day/month/year) 

25/01/1999 


(Earliest) Priority Date (day/month/year) 

25/08/1998 


Applicant 

HOLDING B.E.V. SA et al . 



This International Search Report has been prepared by this International Searching Authority and is transmitted to the applicant 
according to Article 18. A copy is being transmitted to the International Bureau. 

This International Search Report consists of a total of 3 sheets. 

[X| It is also accompanied by a copy of each prior art document cited in this report. 



1 . Basis of the report 

a. With regard to the language, the international search was carried out on the basis of the international application in the 
language in which it was filed, unless otherwise indicated under this item. 

I I the international search was carried out on the basis of a translation of the international application furnished to this 
Authority (Rule 23.1 (b». 



b. 



With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the international search 

was carried out on the basis of the sequence listing ; 

| | contained in the international application in written form. 

filed together with the international application in computer readable form. 

furnished subsequently to this Authority in written form. 

furnished subsequently to this Authority in computer readble form. 



□ 
□ 
□ 
□ 

□ 



□ 
□ 



the statement that the subsequently furnished written sequence listing does not go beyond the disclosure in the 
international application as filed has been furnished. 

the statement that the information recorded in computer readable form is identical to the written sequence listing has been 
furnished 

Certain claims were found unsearchable (See Box I). 
Unity of invention is lacking (see Box II). 



4. With regard to the title, 

fX] the text is approved as submitted by the applicant. 

| | the text has been established by this Authority to read as follows: 



5. With regard to the abstract, 

[X] the text is approved as submitted by the applicant. 

□ the text has been established, according to Rule 38.2(b), by this Authority as it appears in Box III. The applicant may, 
within one month from the date of mailing of this international search report, submit comments to this Authority. 

6. The figure of the drawings to be published with the abstract is Figure No. 15A 7 B 

|~X~| as suggested by the applicant. [X] None of the figures. 

| | because the applicant failed to suggest a figure. 

| | because this figure better characterizes the invention. 
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ternational Application No 
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A. CLASSIFICATION OF SUBJECT MSffER , 

IPC 6 G06T7/00 606K9/46 



According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 

IPC 6 G06T G06K 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practical, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category ° 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


X 


EP 0 380 659 A (KABUSHIKI KAISHA KOMATSU 


79 




SEISAKUSHO) 8 August 1990 




Y 


see claims 1,3,4 


1,33 


Y 


EP 0 394 959 A ( FRAUNHOFER-GESELLSCHAFT 


1,4,21, 




ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG 


33,37,53 




E.V.) 31 October 1990 






see abstract 
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Further documents are listed in the continuation of box C. 



ID 



Patent family members are listed in annex. 



° Special categories of cited documents : 

"A" document defining the general state of the art which is not 

considered to be of particular relevance 
"E" earlier document but published on or after the international 

filing date 

"L" document which may throw doubts on priority claim(s) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

"O" document referring to an oral disclosure, use, exhibition or 
other means 

"P" document published prior to the international filing date but 
later than the priority date claimed 



"T" later document published after the international filing date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

"X" document of particular relevance; the claimed invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document is taken alone 

"Y" document of particular relevance; the claimed invention 

cannot be considered to involve an inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the art. 

"&" document member of the same patent family 



Date of the actual completion of the international search 

21 May 1999 


Date of mailing of the international search report 

28/05/1999 


Name and mailing address of the ISA 

European Patent Office, P.B. 5818 Patentlaan 2 
NL - 2280 HV Rijswijk 
Tel. (+31-70) 340-2040, Tx. 31 651 epo nl, 
Fax: (+31-70) 340-3016 


Authorized officer 

Chateau, J-P 
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C.(Contlnuation) DOCUMENTS CONSIDERED TO BE RELEVANT 



mernational Application No 

T/EP 99/00425 



Category 0 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



YAMADA K ET AL: "IMAGE UNDERSTANDING 
BASED ON EDGE HISTOGRAM METHOD FOR 
REAR-END COLLISION AVOIDANCE SYSTEM" 
PROCEEDINGS OF THE VEHICLE NAVIGATION AND 
INFORMATION SYSTEMS CONFERENCE, YOKOHAMA, 
AUG. 31 - SEPT. 2, 1994,31 August 1994, 
pages 445-450, XP000641348 
INSTITUTE OF ELECTRICAL AND ELECTRONICS 
ENGINEERS 

see page 446, right-hand column, paragraph 
3; figure 3 



24,56,83 



4,21,37, 
53 

65,69, 
71,72 
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T/EP 99/00425 



Patent document 


Publication 


Patent famiiy 


Publication 


cited in search report 


date 


member(s) 


date 



EP 380659 



08-08-1990 



U0 
US 



8903094 A 
5181258 A 



06-04-1989 
19-01-1993 



EP 394959 A 31-10-1990 DE 3913620 A 31-10-1990 

DE 59010833 D 13-08-1998 
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Applicant's or agent's file reference 
300K PCT 362 


See Notification of Transmittal of International 
FOR FURTHER ACTION Preliminary Examination Report (Form PCT/IPEA/416) 


International application No. 
PCT/EP99/00425 


International filing date (day/month/year) 
25/01/1999 


Priority date (day/month/year) 
25/08/1998 


International Patent Classification (IPC) or national classification and IPC 
G06T7/00 


Applicant 

HOLDING B.E.V. SA et al. 



1. This international preliminary examination report has been prepared by this International Preliminary Examining Authority 
and is transmitted to the applicant according to Article 36. 

2. This REPORT consists of a total of 7 sheets, including this cover sheet. 

S This report is also accompanied by ANNEXES, i.e. sheets of the description, claims and/or drawings which have 
been amended and are the basis for this report and/or sheets containing rectifications made before this Authority 
(see Rule 70.1 6 and Section 607 of the Administrative Instructions under the PCT). 

These annexes consist of a total of 1 1 sheets. 



3. This report contains indications relating to the following items: 
I S Basis of the report 



Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations suporting such statement 



II 


□ 


ill 


□ 


IV 


□ 


V 




VI 


□ 


VII 


IS 


VIII 





Certain observations on the international application 



Date of submission of the demand 



16/03/2000 



Date of completion of this report 
29.11.2000 



Name and mailing address of the international 
preliminary examining authority: 
European Patent Office 

D-80298 Munich 
Tel. +49 89 2399 - 0 Tx: 523656 epmu d 

Fax: +49 89 2399 - 4465 



Authorized officer 
Muller,.M 

Telephone No. +49 89 2399 7409 
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INTERNATIONAL PRELIMINARY 
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I. Basis of the report 

1 . This report has been drawn on the basis of (substitute sheets which have been furnished to the receiving Office in 
response to an invitation under Articie 14 are referred to in this report as "originaiiy fiied" and are not annexed to 
the report since they do not contain amendments (Rules 70. 16 and 70.17).): 
Description, pages: 

1 -53 as originally filed 



Claims, No.: 

1-37 as received on 24/08/2000 with letter of 22/08/2000 



Drawings, sheets: 

1-16 as originally filed 



2. With regard to the language, all the elements marked above were available or furnished to this Authority in the 
language in which the international application was filed, unless otherwise indicated under this item. 

These elements were available or furnished to this Authority in the following language: , which is: 

□ the language of a translation furnished for the purposes of the international search (under Rule 23.1 (b)). 

□ the language of publication of the international application (under Rule 48.3(b)). 

□ the language of a translation furnished for the purposes of international preliminary examination (under Rule 
55.2 and/or 55.3). 

3. With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the 
international preliminary examination was carried out on the basis of the sequence listing: 

□ contained in the international application in written form. 

□ filed together with the international application in computer readable form. 

□ furnished subsequently to this Authority in written form. 

□ furnished subsequently to this Authority in computer readable form. 

□ The statement that the subsequently furnished written sequence listing does not go beyond the disclosure in 
the international application as filed has been furnished. 

□ The statement that the information recorded in computer readable form is identical to the written sequence 
listing has been furnished. 

4. The amendments have resulted in the cancellation of: 

□ the description, pages: 

IS the claims, Nos.: 38-91 
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□ the drawings, sheets: 



5. □ Thi^ report has been established as if (some of) the amendments had not been made, since they have been 
considered to go beyond the disclosure as filed (Rule 70.2(c)): 

(Any replacement sheet containing such amendments must be referred to under item 1 and annexed to this 
report.) 



6. Additional observations, if necessary: 



V. Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 

1. Statement 



Novelty (N) Yes: Claims 1 -37 

No: Claims 

Inventive step (IS) Yes: Claims 1-37 (if clarified) 

No: Claims 

Industrial applicability (IA) Yes: Claims 1-37 

No: Claims 



2. Citations and explanations 
see separate sheet 



VII. Certain defects in the international application 

The following defects in the form or contents of the international application have been noted: 
see separate sheet 



VIII. Certain observations on the international application 

The following observations on the clarity of the claims, description, and drawings or on the question whether the 
claims are fully supported by the description, are made: 
see separate sheet 
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Citations 

Reference is made to the following documents: 

D1 : EP-A-0 380 659 (KABUSHIKI KAISHA KOMATSU SEISAKUSHO) 8 August 1 990 
D2: YAMADA K ET AL: 'IMAGE UNDERSTANDING BASED ON EDGE HISTOGRAM 
METHOD FOR REAR-END COLLISION AVOIDANCE SYSTEM' PROC. OF THE 
VEHICLE NAVIGATION AND INFORMATION SYSTEMS CONFERENCE, 
YOKOHAMA, AUG. 31 - SEPT. 2, 1994, pages 445-450, IEEE 

V: Reasoned statement under Rule 66-2(a)(ii) with regard to novelty, inventive 
step or industrial applicability: citations and explanations supporting such 
statement 

1 Under the proviso of the objections raised under item VIII above, the subject 

matter of independent claims 1 . 26 and 33 and, as a consequence, of the depen- 
dent claims 2-25. 27-32 and 34-37 . is considered to meet the requirements of Ar- 
ticle 33 (1-3) PCT. 

1.1 Document D1 represents the closest piece of prior art on file with respect to all 
claims. 

1 .2 D1 discloses process for identifying a line in an input image (page 1 , technical 
field; figures 1a-1e and 2; page 9), the line comprising a plurality of pixels 
corresponding to the line (eg, see abstract, lines 1-2; page 9, 1st par.; fi-gure 18), 
the process including an elementary operation comprising: 

projecting pixels onto a projection axis defined by a rotation angle related to 
a reference axis, as to form a projection waveform (eg, figures 1a-1e); 

analysing the projection waveform to identify characteristics indicative of a 
line (ibid.)] and 

if (and as long as) the analysis of histograms is not indicative of a line, rota- 
ting the projection axis by a predetermined angle (age 9, par. 2 - page 10, par. 2). 

1.3 The fact that claim 1 specifies a histogram where D1 refers to "projection wave- 
forms" is considered insignificant because both closely correspond to each other, 
as the skilled person is well aware. Moreover, it appears that the context of digital 
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image processing actually suggests the (approximative) representation of a 
"projection waveform" by a discrete sampling, ie, a "histogram". 

1.4 D1 does not, however, teach or suggest to perform the line detection operation in 
parallel. 

1 .5 It is held that parallelization as such is well-known and so is the general desire to 
speed up a given process by parallelization. However, it is not considered to be 
obvious to perform N x S parallel operations as specified in claim 1 , in order to 
detect S=2 lines simultaneously and each on one of N>3 predetermined projection 
axes. 

1 .6 By virtue of this feature, hence, the subject matter of present independent claim 1 
is considered to evince an inventive step as required by Article 33 (3) PCT. 

2 The same assessment applies to independent claim 26 under the proviso of the 
clarity problem under item VIII. 1 0.2 and to claim 33 under the proviso of the clarity 
problems under items VIII. 10.2 and 11. 

3 To dependent c laims 2-25, 27-32 and 34-37 applies the conditionally positive 
assessment of their respective independent claim by analogy. 

VII: Certain defects in the international application 

4 Documents D1 and D2 should have been acknowledged in the description and 
their relevant contents should be briefly discussed (Rule 5.1 (a) (ii) PCT). 

5 The two-part form of claim 1 based on document D1 appears to be inappropriate 
with respect to the pixel characteristics (claim page 1 , lines 14-17). 
Histogramming black pixels according to D1 (see, eg, figures 1a-1e) clearly and 
unambiguously corresponds to histogramming pixels of a certain "luminance, hue, 
and/or saturation" (claim page 1 , line 15) for the special case of a black-and-white 

. image (Rule 6.3 (b) (i) PCT). 

6 Reference signs in parentheses should have been inserted in all claims according 



Form PCT/Separate Sheet/409 (Sheet 2) (EPO-April 1997) 



INTERNATIONAL PRELIMINARY International application No. 

EXAMINATION REEORT - SEPARATE SHEET 



PCT/EP99/00425 



to Rule 6.2 (b) PCT. 

7 The phrase on page 16, lines 1-3, which states to "incorporate by reference" the 
contents of preceding PCT applications should have been deleted according to 
the PCT Examination Guidelines 11-4.18. 

8 The PCT application numbers cited on page 1 , section 2, and page 16, 1st and 
2nd par., should have been replaced by corresponding publication numbers. 

9 On page 46, line 4, the wildcard "??" should have been replaced by "25". 
VIII: Certain observations on the international application 

1 0 The three independent claims 1 . 26 and 33 do not correspond to each other to the 
highest possible degree (Article 6 PCT). 

1 0.1 Specifically, both claims 26 and 33 fail to specify a means corresponding to the 
"image processing step used to select pixels" of claim 1 (claim page 1, lines 12- 
17). With respect to claim 26, the corresponding features are specified in claim 29, 
items i) and ii), with respect to claim 33 they are not specified at all. 

10.2 Claims 26 and 33 fail to specify that there are N x S rotation units; requiring 
rotation units "for" S=2 sets and N>3 predetermined projection units does not 
imply their number. Moreover, claims 26 and 33 fail to clearly specify that said 
rotation units are to be operated in parallel (the phrase "through the rotation units" 
on claim page 8, line 18, is considered insufficient for this purpose). 

As said claims stand, the rotation units might just represent redundant 
hardware to ensure fault tolerance - which is neither supported by the description 
nor solves the same technical problem as the subject matter of claim 1 . 

This omission is crucial since inventive step of present claim 1 is conceded 
by virtue of this very feature (cf item V.1 .5 above). 

1 1 Claim 33 is unclear for a number of reasons: 

11.1 The meaning of the terms "class" and "domain" as apparently intended according 
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to claim 33 (cf, description, page 3, lines 1-3, and page 39, par. 2) are not clear 
from the claim, in contradition to the PCT Guidelines C-lll, 4.2. 

11.2 The function of the linear combination unit is not clear from claim 33 as well, nor is 
the function of the "validation signal" precisely clear (claim page 10, lines 6-9). 

Specifically, it is not clear what said linear combination unit actually 
"combines" and how, and also according to which criteria the validation signal 
"selects one or more of the plurality of domains" and for what sort of "processing". 

1 1 .3 It is stressed that these issues should have been clear from the wording of the 
claims alone (Article 6 PCT and PCT Guidelines C-lll, 4.2). 

12 It is not clear from the claims precisely how said pixels are to be selected accor- 
ding to claims 1 and 29 . Specifically, the meaning of DP, CO and velocity a priori 
has no established meaning with respect to a static input image whereas 
luminance, hue and saturation apparently have. 
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Applicant's or agent's file reference 
300K PCT 362 


See Notification of Transmittal of International 
FOR FURTHER ACTION Preliminary Examination Report (Form PCT/IPEA/416) 


International application No. 


International filing date (day/month/year) 


Priority date (day/month/year) 


PCT/EP99/00425 


25/01/1999 


25/08/1998 


International Patent Classification (IPC) or national classification and IPC 
G06T7/00 


Applicant 






HOLDING B.E.V. SA et al. 







1 . This international preliminary examination report has been prepared by this International Preliminary Examining Authority 
and is transmitted to the applicant according to Article 36. 

2. This REPORT consists of a total of 7 sheets, including this cover sheet. 

S This report is also accompanied by ANNEXES, i.e. sheets of the description, claims and/or drawings which have 
been amended and are the basis for this report and/or sheets containing rectifications made before this Authority 
(see Rule 70.16 and Section 607 of the Administrative Instructions under the PCT). 

These annexes consist of a total of 11 sheets. 



3. 



This report contains indications relating to the following items: 
I Kf Basis of the report 



Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations suporting such statement 



II 


□ 


III 


□ 


IV 


□ 


V 




VI 


□ 


VII 




VIII 





Certain observations on the international application 



Date of submission of the demand 



16/03/2000 



Date of completion of this report 
29.11.2000 



Name and mailing address of the international 
preliminary examining authority: 
European Patent Office 

D-80298 Munich 
Tel. +49 89 2399 - 0 Tx: 523656 epmu d 

Fax: +49 89 2399 - 4465 



Authorized officer 
Muller, M 

Telephone No. +49 89 2399 7409 
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I. Basis of the report 

1 . This report has been drawn on the basis of (substitute sheets which have been furnished to the receiving Office in 
response to an invitation under Article 14 are referred to in this report as "originally filed" and are not annexed to 
the report since they do not contain amendments (Rules 70. 16 and 70. 1 7).): 

Description, pages: 

1 -53 as originally filed 

Claims, No.: 

1 -37 as received on 24/08/2000 with letter of 22/08/2000 

Drawings, sheets: 

1-16 as originally filed 

2. With regard to the language, all the elements marked above were available or furnished to this Authority in the 
language in which the international application was filed, unless otherwise indicated under this item. 

These elements were available or furnished to this Authority in the following language: , which is: 

□ the language of a translation furnished for the purposes of the international search (under Rule 23.1 (b)). 

□ the language of publication of the international application (under Rule 48.3(b)). 

□ the language of a translation furnished for the purposes of international preliminary examination (under Rule 
55.2 and/or 55.3). 

3. With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the 
international preliminary examination was carried out on the basis of the sequence listing: 

□ contained in the international application in written form. 

□ filed together with the international application in computer readable form. 

□ furnished subsequently to this Authority in written form. 

□ furnished subsequently to this Authority in computer readable form. 

□ The statement that the subsequently furnished written sequence listing does not go beyond the disclosure in 
the international application as filed has been furnished. 

□ The statement that the information recorded in computer readable form is identical to the written sequence 
listing has been furnished. 

4. The amendments have resulted in the cancellation of: 

D the description, pages: 

E3 the claims, Nos.: 38-91 
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□ the drawings, sheets: 

5. □ This report has been established as if (some of) the amendments had not been made, since they have been 
considered to go beyond the disclosure as filed (Rule 70.2(c)): 

(Any replacement sheet containing such amendments must be referred to under item 1 and annexed to this 
report.) 



6. Additional observations, if necessary: 



V. Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 

1. Statement 



Novelty (N) 


Yes: 


Claims 


1-37 




No: 


Claims 




Inventive step (IS) 


Yes: 


Claims 


1-37 (if clarified) 




No: 


Claims 




Industrial applicability (IA) 


Yes: 


Claims 


1-37 




No: 


Claims 





2. Citations and explanations 
see separate sheet 

VII. Certain defects in the international application 

The following defects in the form or contents of the international application have been noted: 
see separate sheet 

VIII. Certain observations on the international application 

The following observations on the clarity of the claims, description, and drawings or on the question whether the 
claims are fully supported by the description, are made: 
see separate sheet 
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Citations 

Reference is made to the following documents: 

D1 : EP-A-0 380 659 (KABUSHIKI KAISHA KOMATSU SEISAKUSHO) 8 August 1990 
D2: YAMADA K ET AL: 'IMAGE UNDERSTANDING BASED ON EDGE HISTOGRAM 
METHOD FOR REAR-END COLLISION AVOIDANCE SYSTEM' PROC. OF THE 
VEHICLE NAVIGATION AND INFORMATION SYSTEMS CONFERENCE, 
YOKOHAMA, AUG. 31 - SEPT. 2, 1994, pages 445-450, IEEE 

V: Reasoned statement under Rule 66.2(aUiH with regard to novelty, inventive 
step or industrial applicability: citations and explanations supporting such 
statement 

1 Under the proviso of the objections raised under item VIII above, the subject 

matter of independent claims 1 . 26 and 33 and, as a consequence, of the depen- 
dent claims 2-25. 27-32 and 34-37 . is considered to meet the requirements of Ar- 
ticle 33 (1-3) PCT. 

1.1 Document D1 represents the closest piece of prior art on file with respect to all 
claims. 

1.2 D1 discloses process for identifying a line in an input image (page 1 , technical 
field; figures 1a-1e and 2; page 9), the line comprising a plurality of pixels 
corresponding to the line (eg, see abstract, lines 1-2; page 9, 1st par.; fi-gure 18), 
the process including an elementary operation comprising: 

projecting pixels onto a projection axis defined by a rotation angle related to 
a reference axis, as to form a projection waveform (eg, figures 1a-1e); 

analysing the projection waveform to identify characteristics indicative of a 
line (ibid.)\ and 

if (and as long as) the analysis of histograms is not indicative of a line, rota- 
ting the projection axis by a predetermined angle (age 9, par. 2 - page 10, par. 2). 

1 .3 The fact that claim 1 specifies a histogram where D1 refers to "projection wave- 
forms" is considered insignificant because both closely correspond to each other, 
as the skilled person is well aware. Moreover, it appears that the context of digital 
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image processing actually suggests the (approximative) representation of a 
"projection waveform" by a discrete sampling, ie, a "histogram". 

1 .4 D1 does not, however, teach or suggest to perform the line detection operation in 
parallel. 

1 .5 It is held that parallelization as such is well-known and so is the general desire to 
speed up a given process by parallelization. However, it is not considered to be 
obvious to perform N x S parallel operations as specified in claim 1 , in order to 
detect S=2 lines simultaneously and each on one of N>3 predetermined projection 
axes. 

1 .6 By virtue of this feature, hence, the subject matter of present independent claim 1 
is considered to evince an inventive step as required by Article 33 (3) PCT. 

2 The same assessment applies to independent claim 26 under the proviso of the 
clarity problem under item VIII. 10.2 and to claim 33 under the proviso of the clarity 
problems under items VIII. 1 0.2 and 1 1 . 

3 To dependent c laims 2-25. 27-32 and 34-37 applies the conditionally positive 
assessment of their respective independent claim by analogy. 

VII: Certain defects in the international application 

4 Documents D1 and D2 should have been acknowledged in the description and 
their relevant contents should be briefly discussed (Rule 5.1 (a) (ii) PCT). 

5 The two-part form of claim 1 based on document D1 appears to be inappropriate 
with respect to the pixel characteristics (claim page 1 , lines 14-17). 
Histogramming black pixels according to D1 (see, eg, figures 1a-1e) clearly and 
unambiguously corresponds to histogramming pixels of a certain "luminance, hue, 
and/or saturation" (claim page 1, line 15) for the special case of a black-and-white 
image (Rule 6.3 (b) (i) PCT). 

6 Reference signs in parentheses should have been inserted in all claims according 
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to Rule 6.2 (b) PCT. 

7 The phrase on page 16, lines 1-3, which states to "incorporate by reference" the 
contents of preceding PCT applications should have been deleted according to 
the PCT Examination Guidelines 11-4.18. 

8 The PCT application numbers cited on page 1, section 2, and page 16, 1st and 
2nd par., should have been replaced by corresponding publication numbers. 

9 On page 46, line 4, the wildcard "??" should have been replaced by "25". 
Vlll: Certain observations on the international application 

1 0 The three independent claims 1 . 26 and 33 do not correspond to each other to the 
highest possible degree (Article 6 PCT). 

10.1 Specifically, both claims 26 and 33 fail to specify a means corresponding to the 
"image processing step used to select pixels" of claim 1 (claim page 1, lines 12- 
17). With respect to claim 26, the corresponding features are specified in claim 29, 
items i) and ii), with respect to claim 33 they are not specified at all. 

10.2 Claims 26 and 33 fail to specify that there are N x S rotation units; requiring 
rotation units "for" S=2 sets and N>3 predetermined projection units does not 
imply their number. Moreover, claims 26 and 33 fail to clearly specify that said 
rotation units are to be operated in parallel (the phrase "through the rotation units" 
on claim page 8, line 18, is considered insufficient for this purpose). 

As said claims stand, the rotation units might just represent redundant 
hardware to ensure fault tolerance - which is neither supported by the description 
nor solves the same technical problem as the subject matter of claim 1 . 

This omission is crucial since inventive step of present claim 1 is conceded 
by virtue of this very feature (cf item V.1 .5 above). 

1 1 Claim 33 is unclear for a number of reasons: 

11.1 The meaning of the terms "class" and "domain" as apparently intended according 
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to claim 33 (cf, description, page 3, lines 1-3, and page 39, par. 2) are not clear 
from the claim, in contradition to the PCT Guidelines C-lll, 4.2. 

1 1 .2 The function of the linear combination unit is not clear from claim 33 as well, nor is 
the function of the "validation signal" precisely clear (claim page 10, lines 6-9). 

Specifically, it is not clear what said linear combination unit actually 
"combines" and how, and also according to which criteria the validation signal 
"selects one or more of the plurality of domains" and for what sort of "processing". 

1 1 .3 It is stressed that these issues should have been clear from the wording of the 
claims alone (Article 6 PCT and PCT Guidelines C-lll, 4.2). 

12 It is not clear from the claims precisely how said pixels are to be selected accor- 
ding to claims 1 and 29 . Specifically, the meaning of DP, CO and velocity a priori 
has no established meaning with respect to a static input image whereas 
luminance, hue and saturation apparently have. 
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CLAIMS 

1. A process for identifying a line in an input image, the 
image comprising a plurality of pixels corresponding to the line, the 
process including an elementary operation comprising: 

5 - projecting pixels onto a projection axis defined by a 

rotation angle related to a reference axis, as to form a projection 
waveform; 

- analyzing the projection waveform to identify 
characteristics indicative of a line; 

10 characterised in that 

- the projection waveform is an histogram; 

- the image is processed in order to select pixels of the 
image having characteristics corresponding to characteristics of the 
line, the selected pixels being at least from the group consisting of 

15 luminance, hue, saturation, direction, DP, CO and velocity, DP 
identifying a significant variation of a pixel and CO being an 
updated calculated value of time constant; 

- and that N x S elementary operations are done in parallel 
processing with the selected pixels on S =2 sets of N > 3 

20 predetermined projection axis, each set being related to a set 
reference axis {x,y}, the N predetermined projection axis of each 
set being regularly distributed from the set reference axis with a 
stepping angle of 180°/N; 

- and that, if the analysis of the histograms of a set are not 
25 indicative of a line: 

the N predetermined projection axis of said set are rotated 
with a same rotation angle and the elementary operations are done 
until an analysis of the histograms is indicative of a line. 

2. The process according to claim 1 characterised in that the 
30 elementary operations are done in parallel for the S=2 sets, the 

sets being chosen related to {x, y} orthogonal references axis. 

3. The process according to the claim 2 characterised in that 
the references axis are chosen as being the horizontal and vertical 
axis of the image. 
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4. The process according to claim 1 characterised in that the 
elementary operations are done in parallel for the S=2 sets, the 
sets being chosen related to {x, y} non orthogonal references axis. 

5. The process according to any one of claims 1 to 4 
5 characterised in that the elementary operations are done in parallel 

for the S=2 sets of N = 16 predefined projection axis. 

6. The process according to one of claims 1 to 5 
characterised in that it further comprises the identification of the 
orientation of the line, the process further comprising the step of: 

10 if the analysis of the histograms of a set are not indicative of 

a line most closely perpendicular to one of N predetermined 

projection axis of said set: 

the N predetermined projection axis of said set are rotated 

with a same rotation angle and the elementary operations are done 
15 until an analysis of the histograms is indicative of a line most 

closely perpendicular to one of N predetermined projection axis of 

said set. 

7. The process according to claim 6 characterised in that the 
histogram characteristics comprise R =NBPTS/RMAX f and in that 

20 the line is determined to be most closely perpendicular to the 
predetermined projection axis at which R is a minimum, NBPTS 
being the number of points in the histogram and RMAX being the 
number of points at the maximum of the histogram. 

8. The process according to one of the claim 1 to 7 
25 characterised in that a camera is mounted on a vehicle and an 

input image of the road is acquired in order to identify a line on a 
road. 

9. The process according to claim 8 characterised in that the 
line on a road is a broken line and that the step of analyzing the 

30 histogram to identify characteristics indicative of a line comprises 
time averaging the histogram over a succession of frames of the 
image. 

10. The process according to claim 8 characterised in that 
the line on a road is a broken line and that the step of analyzing 

35 the histogram to identify characteristics indicative of a line 
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comprises analyzing the histogram over a succession of frames of 
the image to identify a periodic fluctuation in peaks of the 
histogram indicative of a broken line. 

11. The process according to claim 8 characterised in that 
5 the line is a broken line and that the step of selecting pixels of the 

image having characteristics corresponding to characteristics of the 
line comprises selecting pixels in a first desired area of the image 
for selecting pixels associated with a first portion of the broken 
line, and selecting pixels in a second desired area of the image for 

10 selecting pixels associated with a second portion of the broken line 
adjacent to the first section; and 

that the step forming histograms of the selected pixels 
projected onto a first set of N predetermined axis comprises 
forming first histograms of the selected first pixels projected onto a 

15 first set of N predetermined axis and forming second histograms of 
the selected second pixels projected onto the first set of N 
predetermined axis; and 

that the step of analyzing the histogram to identify 
characteristics indicative of a line comprises analyzing the first and 

20 second histograms over a succession of frames of the image to 
identify a periodic movement of first pixels associated with the line 
from the first desired area to the second desired area. 

12. The process according to claim 8 characterised in that 
the line is a broken line and that the step of selecting pixels of the 

25 image having characteristics corresponding to characteristics of the 
line comprises selecting pixels in a first desired area of the image 
for selecting pixels associated with a first portion of the broken 
line, and selecting pixels in a second desired area of the image for 
selecting pixels associated with a second portion of the broken line 

30 adjacent to the first section; and 

that the step forming histograms of the selected pixels 
projected onto sets of N predetermined axis comprises forming first 
histograms of the selected first pixels projected onto a first set of N 
predetermined axis and forming second histograms of the selected 
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second pixels projected onto a second set of N predetermined axis; 
and 

that the step of analyzing the histograms to identify 
characteristics indicative of a line comprises analyzing the 
5 histograms from first and second sets over a succession of frames 
of the image to identify a periodic movement of first pixels 
associated with the line from the first desired area to the second 
desired area. 

13. The process according to any one of claims 8 to 12 
10 characterised in that the line is a parallel double line and that the 

step of analyzing the histogram to identify characteristics indicative 
of a line comprises analyzing the histogram to identify two peaks 
characteristic of a parallel double line. 

14. The process according to any one of claims 8 to 12 
15 characterised in that a camera is mounted on a vehicle and an 

input image of the road is acquired in order to identify a line on a 
road which is a double line and that the step of selecting pixels of 
the image having characteristics corresponding to characteristics of 
the line comprises selecting pixels in a first desired area of the 

20 image at a first desired orientation in the image for selecting pixels 
associated with the first line, and selecting pixels in a second 
desired area of the image at a second desired orientation in the 
image for selecting pixels associated with the second line; and 

that the step of forming histograms of the selected pixels 

25 projected onto sets of N predetermined axis comprises forming first 
histograms of the selected first pixels projected onto a first set of N 
predetermined axis and forming second histograms of the selected 
second pixels projected onto the first set of predetermined axis; 
and 

30 that the step of analyzing the histograms to identify 

characteristics indicative of a line comprises analyzing each of the 
first and second histograms to identify characteristics indicative of 
a line; and 

that if the analysis of the histograms of the first set are not 
35 indicative of a line: 
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the N predetermined projection axis of said set are rotated 
with a same rotation angle and the elementary operations are done 
until an analysis of the histograms is indicative of a line. 

15. The process according to any one of claims 8 to 12 
5 characterised in that a camera is mounted on a vehicle and an 

input image of the road is acquired in order to identify a line on a 
road which is a double line and that the step of selecting pixels of 
the image having characteristics corresponding to characteristics of 
the line comprises selecting pixels in a first desired area of the 

10 image at a first desired orientation in the image for selecting pixels 
associated with the first line, and selecting pixels in a second 
desired area of the image at a second desired orientation in the 
image for selecting pixels associated with the second line; and 

that the step of forming histograms of the selected pixels 

15 projected onto sets of N predetermined axis comprises forming first 
histograms of the selected first pixels projected onto a first set of N 
predetermined axis and forming second histograms of the selected 
second pixels projected onto a second set of predetermined axis; 
and 

20 that the step of analyzing the histograms to identify 

characteristics indicative of a line comprises analyzing histograms 
of first and second sets to identify characteristics indicative of a 
line; and 

that if the analysis of the histograms of a set are not 
25 indicative of a line: 

the N predetermined projection axis of said set are rotated 
with a same rotation angle and the elementary operations are done 
until an analysis of the histograms is indicative of a line. 

16. The process according to any one of claims 8 to 15 
30 characterised in that a camera is mounted on a vehicle and an 

input image of the road is acquired in order to detect a lane on a 
road from the vehicle-mounted camera, the lane being defined by a 
first line on one side thereof and a second line on the other side 
thereof, the process comprising the steps of: 
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acquiring an image of the road from the camera, each of the 
first and second side lines comprising a plurality of pixels in the 
image; 

selecting pixels of the image in a first desired area of the 
5 image at a first desired orientation in the image for selecting pixels 
associated with the first side line, and selecting pixels in a second 
desired area of the image at a second desired orientation in the 
image for selecting pixels associated with the second side line; 
identifying the first side line and the second side line. 
10 17. The process according to claim 16 characterised in that 

a camera is mounted on a vehicle and an input image of the road is 
acquired in order to detect a vehicle in an adjacent lane from a 
camera mounted to a subject vehicle, the adjacent lane being 
defined by a first line on one side thereof and a second line on the 
15 other side thereof, the process comprising the steps of: 

acquiring an image of the road from the camera, each of the 
first and second side lines comprising a plurality of pixels in the 
image; 

detecting an adjacent lane by identifying the first side line 
20 and the second side line of said adjacent lane; 

selecting pixels of the image having characteristics 
corresponding to characteristics of a vehicle; 

the step of selecting pixels of the image having 
characteristics corresponding to characteristics of a vehicle 
25 comprises selecting such pixels in an area bounded by the first and 
second side lines; 

forming a histogram of the selected pixels projected onto a 
first set of predetermined axis; and 

analyzing the histogram to detect characteristics indicative 
30 of a vehicle. 

18. The process according to claim 17 characterised in that 
the step of analyzing the histogram to detect characteristics 
indicative of a vehicle comprises detecting a histogram having 
NBPTS exceeding a threshold, NBPTS being the number of points 
35 in the histogram. 
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19. The process according to claim 17 or 18 characterised in 
that the step of selecting pixels of the image having characteristics 
corresponding to characteristics of a vehicle comprises selecting 
pixels having a color or luminance characteristic of tail lights . 
5 20. The process according to claim 19 analyzing the 

histogram to detect characteristics indicative of a vehicle comprises 
analyzing the histogram to separately detect each taillight. 

21. The process according to claim 17 or 18 characterised in 
that the step of selecting pixels of the image having characteristics 

10 corresponding to characteristics of a vehicle comprises selecting 
pixels having a color or luminance characteristic of headlights. 

22. The process according to claim 21 analyzing the 
histogram to detect characteristics indicative of a vehicle comprises 
analyzing the histogram to separately detect each headlight. 

15 23. The process according to one of claims 17 to 22 

characterised in that the step of selecting pixels of the image 
having characteristics corresponding to characteristics of a vehicle 
comprises selecting pixels moving in a direction parallel to a 
direction of the lane. 

20 24. The process according to one of claims 17 to 23 

characterised in that the step of selecting pixels of the image 
having characteristics corresponding to characteristics of a vehicle 
comprises selecting pixels moving in a direction generally parallel 
to one of the first or second side lines. 

25 25. The process according to one of claims 17 to 22 

characterised in that it comprises the step of dynamically adapting 
in function of results at least one or more of the following 
parameters: classification, areas, histograms. 

26. An apparatus for identifying the orientation of a line in 

30 an input image, the line comprising a plurality of pixels, said 
apparatus having means for: 

- projecting pixels onto a projection axis defined by a 
rotation angle related to a reference axis, as to form a projection 
waveform; 
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- analyzing the projection waveform to identify 
characteristics indicative of a line; 

characterised in that the projection waveform is an 
histogram and that it comprises means for executing the process of 
5 any one of previous claims, said means being at least an image 
processing system interfaced with a controller: 

- the image processing system comprising histogram 
formation units and rotation units for S = 2 sets of N >3 
predetermined projection axis, said histogram formation units 

10 forming histograms of the pixels projected on the predetermined 
projection axis of sets, and said rotation units rotating by a same 
rotation angle the predetermined projection axis of sets; 

- the controller analyzing the histograms to determine when 
the histograms of a set comprises characteristics indicating that the 

15 line is most closely perpendicular to one of the predetermined axis 
of said set, and 

- the controller being able to rotate the N predetermined axis 
of a set through the rotation units. 

27. The apparatus according to claim 26 characterised in 
20 that the rotation unit comprises a Hough transform unit performing 

a Hough transform on the pixels for enabling rotation of the 
predetermined axis. 

28. The apparatus according to claim 26 or 27 characterised 
in that each histogram formation unit comprises an area 

25 selection memory for selecting an area of an image for which to 
form a histogram, the controller controlling the histogram formation 
unit to select pixels in a desired area of the image for detecting the 
line. 

29. The apparatus according to any one of claims 26 to 28 
30 characterised in that the interface between the image processing 

system and the controller comprises: 

input signals from the controller to the image processing 
system including control signals selected from the group consisting 
of: 
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i) signals for selecting domains for processing by the image 
processing system, said domains being selected from the group 
consisting of luminance, hue, saturation, CO, DP, direction, and 
velocity; 

5 ii) signals for selecting classes of pixels within each domain 

for processing by the image processing system, 

iii) signals for selecting rotation angle of sets for 
formation of histograms projected on the predefined axes of the 
sets, and 

10 iv) signals for selecting an area of an image for processing 

by the image processing system; and 

output signals from the image processing system to the 

controller including signals resultant from processing the input 

signals selected from the group consisting of: 
15 i) signals containing information on histograms formed in the 

image processing system, and 

ii) signals containing histograms formed in the image 
processing system. 

30. The apparatus according to claim 29 characterised in 
20 that the signals containing information on histograms formed in 

the image processing system are selected from the group 
consisting of MIN, MAX, NBPTS, RMAX, POSRMAX, MIN being the 
minimum of an histogram, MAX being the maximum of an 
histogram, POSRMAX being the position of the maximum of an 
25 histogram. 

31. The apparatus according to any one of claims 26 to 
30, characterised in that the apparatus is built in a single chip 
(MOS). 

32. The apparatus according to any one of claims 26 to 31, 
30 characterised in that it further comprises a physical link which is a 

standard automotive bus. 

33. A device for identifying an object in an input signal, the 
object comprising pixels in one of a plurality of classes in one of a 
plurality of domains, the input signal comprising a succession of 
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frames, each frame comprising a succession of pixels, the 
apparatus comprising: 

a classifier (25b) for each domain, the classifier 
classifying pixels within each domain in selected classes within 
5 the domain; 

a linear combination unit for each domain, the linear 
combination unit generating a validation signal for the domain, the 
validation signal selecting one or more of the plurality of domains 
for processing; 

10 rotation units for rotating S=2 sets of N >3 predetermined 

projection axis; 

histogram formation units for forming histograms for 
pixels of the output signal within the classes selected by the 
classifier within each domain selected by the validation signal 
15 projected onto sets of predetermined projection axis; and 

a controller for controlling the classifier, linear combination 
unit, rotation unit, and histogram formation unit for identifying the 
object. 

34. The device according to claim 33 wherein the rotation 
20 unit performs a Hough transform. 

35. The device according to claim 33 or 34 wherein the 
rotation units enable the rotation of a first set of predetermined 
axis and of a second set of predetermined axis, and wherein the 
histogram formation units are capable of forming first histograms 

25 projected onto the first set of predetermined axis, and of forming 
second histograms projected onto the second set of predetermined 
axis. 

36. The device according to any one of claims 33 to 35 
characterised in that the object is in an area of the image, 

30 and the device further comprises an area selection unit for 
selecting an area of the image, the histogram formation units 
forming histograms for pixels of the input signal within the 
selected area of the image, and the controller further controls the 
area selection unit for enabling selection of objects in a desired 

35 area of an image. 
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37. The device according to any one of claims 33 to 36 
characterised in that the object is in an area of the image, and the 
device further comprises a masking unit for masking an area of the 
image to prevent consideration of the pixels in the masked area, 
5 and the controller further controls the masking unit for identifying 
the object. 
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(57) Abstract 

In a process for identifying the orientation of a line in an image, 
a histogram is formed of pixels of the line projected onto an axis. The 
axis is rotated and a histogram is formed of the pixels projected onto 
the rotated axis until the histogram includes charateristics indicating 
that the line is most closely perpendicular to the rotated axis. In a 
process of detecting a line on a road, an image is acquired of the 
road, and pixels of the image having characteristics corresponding 
to characteristics of the line are selected. A histogram is formed of 
the selected pixels projected onto an axis. The axis is rotated and a 
histogram formed of the selected pixels projected onto the rotated axis 
until the histogram includes characteristics indicative of a line. In a 
process of detecting a lane on a road having left and righ side lines, 
pixels of the image in a first area of the image at a first orientation 
and pixels in a second area of the image at a second orientation 
in the image are selected for selecting pixels associated with the 
lines. Histograms are formed projected on the first and second 
axes, respectively, and the axes are rotated until each histogram 
includes characteristics of a line. Also disclosed is a process of 
detecting a vehicle in an adjacent lane, systems for performing the 
aforementioned processes, systems for identifying an object and an 
input signal, and an interface between an image processing system 
and a controller. 
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CLAIMS 

1. A process for identifying the orientation of a line in an input 
image, the line comprising a plurality of pixels, the process comprising the steps of: 

forming a histogram of the pixels projected onto a first axis; 

rotating the first axis and forming a histogram of the pixels projected 
onto the rotated first axis until the histogram comprises characteristics indicating that the 
line is most closely perpendicular to the rotated first axis. 

2. The process according to claim 1 wherein the histogram 
characteristics comprise R =NBPTS/RMAX, and wherein the line is determined to be 
most closely perpendicular to the rotated first axis at which R is a minimum. 

3. The process according to claim 1 wherein the first axis is the 
horizontal or vertical axis of the image. 

4. A process of detecting a line on a road from a vehicle-mounted 
camera, the process comprising the steps of: 

acquiring an image of the road, the image comprising a plurality of pixels 
corresponding to the line; 

selecting pixels of the image having characteristics corresponding to 
characteristics of the line; 

forming a histogram of the selected pixels projected onto a first axis; 

analyzing the histogram to identify characteristics indicative of a line; 

rotating the first axis and forming a histogram of the selected pixels 
projected onto the rotated first axis until the histogram comprises characteristics 
indicative of a line. 

5. The process according to claim 4 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a line 
comprises selecting pixels selected from the group consisting of luminance, hue, 
saturation, direction, DP, CO and velocity. 

6. The process according to claim 4 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a line 
comprises selecting pixels in a desired area of the image. 
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7. The process according to claim 4 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a line 
comprises selecting pixels in a desired area of the image at a desired orientation in the 
image. 

8. The process according to claim 4 further comprising repeating the 
step of rotating the first axis and forming a histogram of the selected pixels until the 
histogram comprises characteristics indicating that the line is most closely perpendicular 
to the rotated first axis. 

9. The process according to claim 8 wherein the histogram 
characteristics comprise R =NBPTS/RMAX, and wherein the line is determined to be 
most closely perpendicular to the rotated first axis at the rotated first axis at which R is a 
minimum. 

10. The process according to claim 8 wherein the first axis is the 
horizontal or vertical axis of the image. 

1 1 . The process according to claim 4 wherein the line is a double line 

and wherein: 

the step t of selecting pixels of the image having characteristics 
corresponding to characteristics of the line comprises selecting first pixels in a first 
desired area of the image at a first desired orientation in the image for selecting pixels 
associated with the first line, and selecting pixels in a second desired area of the image at 
a second desired orientation in the image for selecting pixels associated with the second 
line; 

the step of forming a histogram of the selected pixels projected onto a 
first axis comprises forming a first histogram of the selected first pixels projected onto a 
first axis and forming a second histogram of the selected second pixels projected onto the 
first axis; 

the step of analyzing the histogram to identify characteristics indicative of 
a line comprises analyzing each of the first and second histograms to identify 
characteristics indicative of a line; and 
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the step of rotating the first axis comprises rotating the first axis and 
forming first and second histograms of the selected first and second pixels respectively 
projected onto the rotated first axis until each of the first and second histograms 
comprises characteristics indicative of a line. 
5 12. The process according to claim 11 further comprising repeating 

the step of rotating the first axis and forming first and second histogram until each of the 
first and second histograms comprises characteristics indicative of a line and until at least 
one of the first and second histograms comprises characteristics indicating that the line 
associated with such histogram is most closely perpendicular to the rotated first axis. 
10 13, The process according to claim 12 wherein the histogram 

characteristics comprise R =NBPTS/RMAX, and wherein the line is determined to be 
most closely perpendicular to the rotated first axis at the rotated first axis at which R is a 
minimum. 

14. The process according to claim 11 wherein the double lines are 
15 parallel to the other and each line is selected from the group consisting of solid and 

broken lines. 

15. The process according to claim 4 wherein the line is a parallel 
double line and wherein the step of analyzing the histogram to identify characteristics 
indicative of a line comprises analyzing the histogram to identify two peaks characteristic 

20 of a parallel double line. 

16. The process according to claim 4 wherein the line is solid or 

broken. 

17. The process according to claim 4 wherein the line is a broken line 

and wherein; 

25 the step of analyzing the histogram to identify characteristics indicative of 

a line comprises time averaging the histogram over a succession of frames of the image. 

18. The process according to claim 4 wherein the line is a broken line 

and wherein: 
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the step of analyzing the histogram to identify characteristics indicative of 
a line comprises analyzing the histogram over a succession of frames of the image to 
identify a periodic fluctuation in peaks of the histogram indicative of a broken line. 

19. The process according to claim 4 wherein the line is a broken line 

and wherein: 

the step of selecting pixels of the image having characteristics 
corresponding to characteristics of the line comprises selecting first pixels in a first 
desired area of the image for selecting pixels associated with a first portion of the broken 
line, and selecting pixels in a second desired area of the image for selecting pixels 
associated with a second portion of the broken line adjacent to the first section; 

the step forming a histogram of the selected pixels projected onto a first 
axis comprises forming a first histogram of the selected first pixels projected onto a first 
axis and forming a second histogram of the selected second pixels projected onto the first 
axis; and 

the step of analyzing the histogram to identify characteristics indicative of 
a line comprises analyzing the first and second histograms over a succession of frames of 
the image to identify a periodic movement of first pixels associated with the line from the 
first desired area to the second desired area. 

20. The process according to claim 1 1 further comprising repeating 
the step of rotating the first axis and forming first and second histograms until each of 
the first and second histograms comprises characteristics indicative of a line and until at 
least one of the first and second histograms comprises characteristics indicating that the 
line associated with such histogram is most closely perpendicular to the rotated first axis. 

21. A process of detecting a lane on a road from a vehicle-mounted 
camera, the lane being defined by a first line on one side thereof and a second line on the 
other side thereof, the process comprising the steps of: 

acquiring an image of the road from the camera, each of the first and 
second lines comprising a plurality of pixels in the image; 

selecting pixels of the image in a first desired area of the image at a first 
desired orientation in the image for selecting pixels associated with the first line, and 
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selecting pixels in a second desired area of the image at a second desired orientation in 
the image for selecting pixels associated with the second line; 

forming a first histogram of the selected first pixels projected onto a first 
axis and forming a second histogram of the selected second pixels projected onto a 
second axis; 

analyzing each of the first and second histograms to identify 
characteristics in each histogram indicative of a line; and 

until the first histogram comprises characteristics indicative of a line, 
rotating the first axis and forming a first histogram of the first pixels projected onto the 
rotated first axis, and until the second histogram comprises characteristics indicative of a 
line, rotating the second axis and forming a second histogram of the second pixels 
projected onto the rotated second axis. 

22. The process according to claim 21 further comprising repeating 
the step of rotating the first axis and forming a first histogram and rotating the second 
axis and forming a second histogram until each of the first and second histograms 
comprises characteristics indicative of a line and until at least one of the first and second 
histograms comprises characteristics indicating that the line associated with such 
histogram is most closely perpendicular to the rotated first axis. 

23. The process according to claim 22 wherein the histogram 
characteristics comprise R =NBPTS/RMAX, and wherein the line is determined to be 
most closely perpendicular to the rotated first axis at the rotated first or second axis at 
which R is a minimum. 

24. A process for detecting a vehicle in an adjacent lane from a 
camera mounted to a subject vehicle, the process comprising the steps of: 

acquiring an image of the adjacent lane; 

selecting pixels of the image having characteristics corresponding to 
characteristics of a vehicle; 

forming a histogram of the selected pixels projected onto a first axis; and 
analyzing the histogram to detect characteristics indicative of a vehicle. 



SUBSTITUTE SHEET (RULE 26) 



WO 00/11609 




PCT/EP99/00425 



49 



25. The process according to claim 24 wherein the adjacent lane is 
defined by first and second side lines each of the first and second side lines comprising a 
plurality of pixels in the image, wherein: 

the step of acquiring an image of the adjacent lane comprises: 
i) selecting pixels of the image in a first desired area of the image at a 
first desired orientation in the image for selecting pixels associated with the first line, and 
selecting pixels in a second desired area of the image at a second desired orientation in 
the image for selecting pixels associated with the second line; 

ii) forming a first histogram of the selected first pixels projected onto a 
first axis and forming a second histogram of the selected second pixels projected onto a 
second axis; 

iii) analyzing each of the first and second histograms to identify 
characteristics in each histogram indicative of a line; and 

iv) until the first histogram comprises characteristics indicative of a line, 
rotating the first axis and forming a first histogram of the first pixels projected onto the 
rotated first axis, and until the second histogram comprises characteristics indicative of a 
line, rotating the second axis and forming a second histogram of the second pixels 
projected onto the rotated second axis; and 

the step of selecting pixels of the image having characteristics 
corresponding to characteristics of a vehicle comprises selecting such pixels in an area 
bounded by the first and second side lines. 

26. The process according to claim 24 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a vehicle 
comprises selecting pixels selected from the group consisting of luminance, hue, 
saturation, DP, velocity and direction. 

27. The process according to claim 24 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a vehicle 
comprises selecting pixels having a color or luminance characteristic of taillights. 
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28. The process according to claim 27 analyzing the histogram to 
detect characteristics indicative of a vehicle comprises analyzing the histogram to 
separately detect each taillight. 

29. The process according to claim 24 wherein the step of selecting 
5 pixels of the image having characteristics corresponding to characteristics of a vehicle 

comprises selecting pixels having color or luminance characteristics of headlights. 

30. The process according to claim 24 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a vehicle 
comprises selecting pixels moving in a direction parallel to a direction of the lane. 

10 31. The process according to claim 25 wherein the step of selecting 

pixels of the image having characteristics corresponding to characteristics of a vehicle 
comprises selecting pixels moving in a direction generally parallel to one of the first or 
second side lines. 

32. The process according to claim 24 wherein the step of analyzing 
15 the histogram to detect characteristics indicative of a vehicle comprises detecting a 

histogram having NBPTS exceeding a threshold. 

33. An apparatus for identifying the orientation of a line in an input 
image, the line comprising a plurality of pixels, the apparatus comprising: 

a histogram formation unit for forming a histogram of the pixels projected 
20 onto a first axis; and 

a controller for selectively rotating the first axis, the histogram formation 
unit forming a histogram of the pixels projected onto the rotated first axis, the controller 
analyzing the histogram to determine when the histogram comprises characteristics 
indicating that the line is most closely perpendicular to the rotated first axis. 
25 34. The apparatus according to claim 33 wherein the histogram 

formation unit comprise a Hough transform unit performing a Hough transiform on the 
pixels for enabling rotation of the first axis. 

35. The apparatus according to claim 33 wherein the histogram 
formation unit computes R =NBPTS/RMAX, and wherein the controller determines the 
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rotated first axis at which R is a minimum to identify the line most closely perpendicular 
to the rotated first axis. 

36. The apparatus according to claim 33 wherein the first axis is the 
horizontal or vertical axis of the image. 

37. An apparatus for detecting a line on a road, which comprises: 

a vehicle-mounted camera acquiring an image of the road, the line 
comprising a plurality of pixels in the image; 
a controller; and 

a histogram formation unit for forming a histogram on pixels having 
selected characteristics on a selected axis, 

the controller controlling the histogram formation unit to select pixels of 
the image having characteristics corresponding to characteristics of a line and to form a 
histogram projected onto a first axis, the controller analyzing the histogram to identify 
characteristics indicative of a line, the controller further rotating the first axis and 
controlling the histogram formation unit to form a histogram projected onto the rotated 
first axis and analyzing the histogram until the histogram comprises characteristics 
indicative of a line. 

38. The apparatus according to claim 37 wherein the selected 
characteristics are selected from the group consisting of luminance, hue, saturation, 
direction, DP, CO and velocity. 

39. The apparatus according to claim 37 wherein the histogram 
formation unit comprises an area selection memory for selecting an area of an image for 
which to form a histogram, the controller controlling the histogram formation unit to 
select pixels in a desired area of the image for detecting the line. 

40. The apparatus according to claim 37 wherein the histogram 
formation unit comprises an area selection memory for selecting an area of an image for 
which to form a histogram and an angle selection memory for selecting an orientation 
angle for forming a histogram, the controller controlling the histogram formation unit to 
select pixels in a desired area of the image and to form a histogram at a desired 
orientation angle for detecting the line. 
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41. The apparatus according to claim 40 wherein the histogram 
formation unit comprise a Hough transform unit performing a Hough transform on the 
pixels for enabling rotation of the first axis for selecting pixels at the desired orientation 
angle. 

5 42. The apparatus according to claim 37 wherein the controller 

rotates the first axis and the histogram formation unit forms a histogram of the selected 
pixels until the controller determines that the histogram comprises characteristics 
indicating that the line is most closely perpendicular to the rotated first axis. 

43. The apparatus according to claim 42 wherein the histogram 
10 formation unit computes R =NBPTS/RMAX, and wherein the controller determines the 

rotated first axis at which R is a minimum to identify the line most closely perpendicular 
to the rotated first axis. 

44. The apparatus according to claim 37 wherein the line is a double 
line and wherein the histogram formation unit comprises an area selection memory for 

15 selecting an area of an image for which to form a histogram and an angle selection 
memory for selecting an orientation angle for forming a histogram, 

the controller controlling the histogram formation unit for selecting pixels 
of the image having characteristics corresponding to characteristics of a line in a first 
desired area of the image at a first desired orientation in the image for selecting pixels 

20 associated with a first line of the double line and controlling the histogram formation unit 
for selecting pixels in a second desired area of the image at a second desired orientation 
in the image for selecting pixels associated with the second line, 

the histogram formation unit forming a first histogram of the selected first 
pixels projected onto the first axis and forming a second histogram of the selected second 

25 pixels projected onto the first axis, the controller analyzing each of the first and second 
histograms to identify characteristics indicative of a line and rotating the first axis until 
each of the first and second histograms comprises characteristics indicative of a line. 

45. The apparatus according to claim 44 wherein the controller 
rotates the first axis until each of the first and second histograms comprises 

30 characteristics indicative of a line and until at least one of the first and second histograms 
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comprises characteristics indicating that the line associated with such histogram is most 
closely perpendicular to the rotated first axis. 

46. The apparatus according to claim 44 wherein the double lines are 
parallel to the other and each line is selected from the group consisting of solid and 
broken lines. 

47. The apparatus according to claim 37 wherein the line is a parallel 
double line and wherein the controller analyzes the histogram to identify two peaks 
characteristic of a parallel double line. 

48. The apparatus according to claim 37 wherein the line is solid or 

broken. 

49. The apparatus according to claim 37 wherein the line is a broken 
line and wherein the controller time averages the histogram over a succession of frames 
of the image to identify characteristics indicative of a broken line. 

50. The apparatus according to claim 37 wherein the line is a broken 
line and wherein the controller time analyzes the histogram over a succession of frames 
of the image to identify a periodic fluctuation in peaks of the histogram indicative of a 
broken line. 

51. The apparatus according to claim 37 wherein the line is a broken 
line and wherein the controller controls the histogram formation unit to form a first 
histogram of first pixels in a first desired area of the image associated with a first portion 
of the broken line, and to form a second histogram of second pixels in a second desired 
area of the image associated with a second portion of the broken line adjacent to the first 
section, each of the first and second histograms being projected onto the first axis, the 
controller analyzing the first and second histograms over a succession of frames of the 
image to identify a periodic movement of first pixels associated with the line from the 
first desired area to the second desired area. 

52. The apparatus according to claim 44 wherein the controller 
rotates the first axis for forming first and second histogram until each of the first and 
second histograms comprises characteristics indicative of a line and until at least one of 
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section, each of the first and second histograms being projected onto the first axis, the 
controller analyzing the first and second histograms over a succession of frames of the 
image to identify a periodic movement of first pixels associated with the line from the 
first desired area to the second desired area. 
5 52. The apparatus according to claim 44 wherein the controller 

rotates the first axis for forming first and second histogram until each of the first and 
second histograms comprises characteristics indicative of a line and until at least one of 
the first and second histograms comprises characteristics indicating that the line 
associated with such histogram is most closely perpendicular to the rotated first axis. 
10 53. An apparatus for detecting a lane on a road, the lane being defined 

by a first line on one side thereof and a second line on the other side thereof, the 
apparatus comprising: 

a vehicle-mounted camera for acquiring an image of the road, each of the 
first and second lines comprising a plurality of pixels in the image; 
15 a controller; and 

a histogram formation unit for selecting pixels in an image having 
particular characteristics and for forming a histogram of the selected pixels, 

the controller controlling the histogram formation unit for selecting pixels 
of the image in a first desired area of the image and for forming a histogram of the 
20 selected pixels in the first desired area projected onto a first axis for forming a first 
histogram of pixels associated with the first line, and further controlling the histogram 
formation unit for selecting pixels in a second desired area of the image and for forming a 
second histogram of the selected pixels in the second desired area projected onto a 
second axis for forming a histogram of pixels associated with the second line, 
25 the controller analyzing each of the first and second histograms to identify 

characteristics in each histogram indicative of a line, and 

the controller rotating the first axis until the first histogram comprises 
characteristics indicative of a line, and rotating the second axis until the second 
histogram comprises characteristics indicative of a line. 
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54. The apparatus according to claim 53 wherein the controller 
rotates the first axis and the second axis until each of the first and second histograms 
comprises characteristics indicative of a line and until at least one of the first and second 
histograms comprises characteristics indicating that the line associated with such 

5 histogram is most closely perpendicular to the rotated first axis. 

55. The apparatus according to claim 53 wherein the histogram 
formation unit computes R =NBPTS/RMAX for each of the first and second histograms, 
and wherein the controller determines the rotated first axis at which R is a minimum to 
determine when the first line is most closely perpendicular to the rotated first axis, and 

1 0 wherein the controller determines the rotated second axis at which R is a minimum to 
determine when the second line is most closely perpendicular to the rotated second axis. 

56. An apparatus for detecting a vehicle in an adjacent lane from a 
subject vehicle, the apparatus comprising: 

a camera mounted to the subject vehicle for acquiring an image of the 

15 adjacent lane; 

a histogram formation unit for selecting pixels of the image and for 
forming a histogram of such images; and 

a controller for controlling the histogram formation unit to select pixels 
having characteristics corresponding to characteristics of a vehicle and for analyzing the 
20 histogram of such pixels to detect characteristics indicative of a vehicle. 

57. The apparatus according to claim 56 wherein the adjacent lane is 
defined by first and second side lines, each of the first and second side lines comprising a 
plurality of pixels in the image, and wherein: 

the controller controls the histogram formation unit for i) selecting pixels 
25 of the image in a first desired area of the image at a first desired orientation in the image 
for selecting pixels associated with the first side line, and selecting pixels in a second 
desired area of the image at a second desired orientation in the image for selecting pixels 
associated with the second side line, and ii) forming a first histogram of the selected first 
pixels projected onto a first axis and forming a second histogram of the selected second 
30 pixels projected onto a second axis; 
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the controller analyzes each of the first and second histograms to identify 
characteristics in each histogram indicative of a line; 

until the first histogram comprises characteristics indicative of a line, the 
controller rotates the first axis and controls the histogram formation unit to form a first 
5 histogram of the first pixels projected onto the rotated first axis, and until the second 
histogram comprises characteristics indicative of a line, the controller rotates the second 
axis and causes the histogram formation unit to form a second histogram of the second 
pixels projected onto the rotated second axis; and 

the controller controlling the histogram formation unit to select pixels of 
10 the image having characteristics corresponding to characteristics of a vehicle comprises 
in an area bounded by the first and second side lines. 

58. The apparatus according to claim 56 wherein the pixel 
characteristics corresponding to characteristics of a vehicle are selected from the group 
consisting of luminance, hue, saturation, DP, velocity and direction. 
15 59. The apparatus according to claim 56 wherein the controller 

controls the histogram formation unit to select pixels having a color or luminance 
characteristic of taillights. 

60. The apparatus according to claim 59 wherein the controller 
analyzes the histogram to separately detect each taillight. 
20 61. The apparatus according to claim 56 wherein the controller 

controls the histogram formation unit to select pixels having color or luminance 
characteristics of headlights. 

62. The apparatus according to claim 56 wherein the controller 
controls the histogram formation unit to select pixels moving in a direction parallel to a 

25 direction of the lane. 

63. The process according to claim 57 wherein the controller controls 
the histogram formation unit to select pixels moving in a direction generally parallel to 
one of the first or second side lines. 

64. The process according to claim 56 wherein the controller detects a 
30 histogram having NBPTS exceeding a threshold in order to identify a vehicle. 
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65. An apparatus for identifying an object in an input signal, the object 
comprising pixels in one of a plurality of classes in one of a plurality of domains, the 
input signal comprising a succession of frames, each frame comprising a succession of 
pixels, the apparatus comprising: 

a classifier for each domain, the classifier classifying pixels within 
each domain in selected classes within the domain; 

a linear combination unit for each domain, the linear combination unit 
generating a validation signal for the domain, the validation signal selecting one or more 
of the plurality of domains for processing; 

a rotation unit for enabling selection of a histogram formation axis; 

a histogram formation unit for forming a histogram for pixels of the 
output signal within the classes selected by the classifier within each domain selected 
by the validation signal projected onto the histogram formation axis; and 

a controller for controlling the classifier, linear combination unit, rotation 
unit, and histogram formation unit for identifying the object. 

66. The apparatus according to claim 65 wherein the rotation unit 
performs a Hough transform. 

67. The apparatus according to claim 65 wherein the rotation unit 
enables selection of a first histogram formation axis and a second histogram formation 
axis, and wherein the histogram formation unit is capable of forming a first histogram 
projected onto the first histogram formation axis, and of forming a second histogram 
projected onto the second histogram formation axis. 

68. The apparatus according to claim 65 wherein the object is in an 
area of the image, and further comprising an area selection unit for selecting an area of 
the image, the histogram formation unit forming a histogram for pixels of the output 
signal within the selected area of the image, and the controller controlling the area 
selection unit for enabling selection of objects in a desired area of an image. 

69. An apparatus for identifying an object in an input signal, the object 
comprising pixels in an area of the image in one of a plurality of classes in one of a 
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plurality of domains, the input signal comprising a succession of frames, each frame 
comprising a succession of pixels, the apparatus comprising: 

a classifier for each domain, the classifier classifying pixels within 
each domain in selected classes within the domain; 

a linear combination unit for each domain, the linear combination unit 
generating a validation signal for the domain, the validation signal selecting one or more 
of the plurality of domains for processing; 

an area selection unit for selecting an area of the image; 

a histogram formation unit for forming a histogram for pixels of the 
output signal within the selected area of the image within the classes selected by the 
classifier within each domain selected by the validation signal projected onto the 
histogram formation axis; and 

a controller for controlling the classifier, linear combination unit, area 
selection unit, and histogram formation unit for identifying the object. 

70. The apparatus according to claim 69 further comprising a rotation 
unit for enabling selection of a histogram formation axis, the histogram formation unit 
forming a histogram for pixels of the output signal projected onto the histogram 
formation axis. 

71 . An apparatus for identifying an object in an input signal, the object 
comprising pixels in an area of the image in one of a plurality of classes in one of a 
plurality of domains, the input signal comprising a succession of frames, each frame 
comprising a succession of pixels, the apparatus comprising: 

a classifier for each domain, the classifier classifying pixels within 
each domain in selected classes within the domain, 

a linear combination unit for each domain, the linear combination unit 
generating a validation signal for the domain, the validation signal selecting one or more 
of the plurality of domains for processing; 

a masking unit for masking an area of the image to prevent consideration 
of the pixels in the masked area; 
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a histogram formation unit for forming a histogram for pixels of the 
output signal outside the masked area but within the classes selected by the classifier and 
within each domain selected by the validation signal projected onto the histogram 
formation axis; and 

5 a controller for controlling the classifier, linear combination unit, masking 

unit, and histogram formation unit for identifying the object. 

72. An interface between an image processing system and a controller, 
the interface comprising: 

input signals from the controller to the image processing system including 
10 control signals selected from the group consisting of: 

i) signals for selecting domains for processing by the image processing 

system, 

ii) signals for selecting classes of pixels within each domain for processing 
by the image processing system, 

15 iii) signals for selecting axes for formation of histograms projected on the 

selected axes, and 

iv) signals for selecting an area of an image for processing by the image 
processing system; and 

output signals from the image processing system to the controller 
20 including signals resultant from processing the input signals selected from the group 
consisting of: 

i) signals containing information on histograms formed in the image 
processing system, and 

ii) signals containing histograms formed in the image processing system. 
25 73. The interface according to claim 72 wherein the domains are 

selected from the group consisting of luminance, hue, saturation, CO, DP, direction, and 
velocity. 

74. The interface according to claim 72 wherein the signals containing 
information on histograms formed in the image processing system are selected from the 
30 group consisting of MIN, MAX, NBPTS, RMAX, POSRMAX. 
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75. An apparatus according to claim 53, wherein the apparatus is built 
in a single chip (MOS). 

76. An interface according to claim 72, wherein the physical link is a 
standard automotive bus. 

5 77. A process according to claim 1, wherein it is possible to 

dynamically adapt in function of results at least one or more of the following parameters: 
classification, areas, histograms. 

78. Use of an interface according to claim 72 with a physical link 
according to claim 76 and according to claim 77. 
10 79. A process for identifying the orientation of a line in an input 

image, the line comprising a plurality of pixels, the process comprising the steps of: 
forming histograms of the pixels projected onto multiple axes; 
process until the histrograms comprise characteristics indicating that the 
line is most closely perpendicular to one of the multiple axes. 
15 80. The process according to claim 79, wherein the histograms 

characteristics comprise R = NBPTS/RMAX, and wherein the line is determined to be 
most closely perpendicular to one of the multiple axes at which R is a minimum. 

81. The process according to claim 79, wherein one of the multiple 
axes is the horizontal or vertical axis of the image. 
20 82. The process according to claim 79, wherein processing is done 

using parallel computation. 

83. A process of detecting a line on a road from a vehicle-mounted 
camera, the process comprising the steps of: 

acquiring an image of the road, the image comprising a plurality of pixels 
25 corresponding to the line; 

selecting pixels of the image having characteristics corresponding to 
characteristics of the line; 

forming histograms of the selected pixels projected onto multiple axes; 
analysing the histograms to identify characteristics indicative of a line; 
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process until one of the histograms comprises characteristics indicative of 

a line. 

84. The process according to claim 83 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a line 

5 comprises selecting pixels selected from the group consisting of luminance, hue, 
saturation, direction, DP, CO and velocity. 

85. The process according to claim 83, wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a line 
comprises selecting pixels in a desired area of the image. 

10 86. The process according to claim 83, wherein the step of selecting 

pixels of the image having characteristics corresponding to characteristics of a line 
comprises selecting pixels in a desired area of the image at a desired orientation in the 
image. 

87. The process according to claim 83, further comprising repeating 
1 5 the step of rotating the multiple axes and forming histograms of the selected pixels until 

one of the histograms comprises characteristics indicting that the line is most closely 
perpendicular to one of the multiple axes. 

88. The process according to claim 87, wherein the histogram 
characteristics comprise R = NPBTS/RMAX, and wherein the line is determined to be 

20 most closely perpendicular to one of the multiple axes at one of the multiple axes at 
which R is a minimum. 

89. The process according to claim 87, wherein one of the multiple 
axes is the horizontal or vertical axes of the image. 

90. Process according to claim 87, wherein processing is done using 
25 parallel computation. 

91. A process according to claim 79, comprising the step of 
dynamically adapting in function of the results at least one of the following parameters: 
classification, areas, histograms. 
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(54) Title: IMAGE PROCESSING APPARATUS AND METHOD 
(57) Abstract 

In a process for identifying the orientation of a line in an image, 
a histogram is formed of pixels of the line projected onto an axis. The 
axis is rotated and a histogram is formed of the pixeis projected onto 
the rotated axis until the histogram includes charateristics indicating 
that the line is most closely perpendicular to the rotated axis. In a 
process of detecting a line on a road, an image is acquired of the 
road, and pixels of the image having characteristics corresponding 
to characteristics of the line are selected. A histogram is formed of 
the selected pixels projected onto an axis. The axis is rotated and a 
histogram formed of the selected pixels projected onto the rotated axis 
until the histogram includes characteristics indicative of a line. In a 
process of detecting a lane on a road having left and righ side lines, 
pixels of the image in a first area of the image at a first orientation 
and pixels in a second area of the image at a second orientation 
in the image are selected for selecting pixels associated with the 
lines. Histograms are formed projected on the first and second 
axes, respectively, and the axes are rotated until each histogram 
includes characteristics of a line. Also disclosed is a process of 
detecting a vehicle in an adjacent lane, systems for performing the 
aforementioned processes, systems for identifying an object and an 
input signal, and an interface between an image processing system 
and a controller. 
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